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Abstract 


Variety, size and complexity of data types, services and applications in Internet is continuously growing up. This increasing of 
complexity needs more powerful and sophisticated equipment’s. One group of these devices that has essential role are routers. 
Some of vendors produce some elaborate and complex products but the commercial solutions are too closed and inflexible. The 
term Open Source Routers covers a lot of implementations of free software routers. Open Source Routers are solutions to overcome 
commercial solutions with closed platforms. In this article, we survey the existing implementations and a wide array of past and 
state-of-the-art projects on open software routers followed by a discussion of major challenges in this area. 
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1. Introduction 


Software router field is not a new topic. Many efforts has 
been made in this area and currently there are some research 
groups who involved in the implementation and further expan- 
sion of these open source projects. It is interesting to know that 
some of these projects are supported by well-known compa- 
nies such as Microsoft, Intel and AT&T. For instance, XORP is 
supported by Intel Corporation and the National Science Foun- 
dation institute [1]. Closeness and lack of the ability to develop 
a product by anyone other than the manufacturer is a major rea- 
son for the formation and emersion of software routers, in addi- 
tion to the high cost of hardware routers. On the other hand, one 
company’s production is rarely compatible with the products 
provided by other companies. The main idea of implement- 
ing these software routers is to use general-purpose computers 
as the hardware platform and design software routines to man- 
age data packets through the forwarding and routing processes 
in a complete open source format. The main platform which 
development of software is performed on, is the open source 
Linux operating system that acts as a powerful platform. This 
networking field is evolving and has been lead toward develop- 
ment in a way that its impact can vividly be seen in different 
environments. For example those small and inexpensive de- 
vices which are used in small offices in order to connect to the 
Internet, are presented in low prices and good performance are 
all this movement’s production. Different versions of the soft- 
ware has become commercial in particular names and are em- 
bedded in various hardware products. TomatoUSB (modified 
Tomato firmware) [2] is a good instance in this regard. As is 
shown, there are a lot of benefits that can be envisioned in these 
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types of routers. Among low cost of the product, open source 
feature, the compatibility with other similar products, various 
manufacturers, continuous updating and development [3]. But 
in the meantime, there are some drawbacks. One of the main 
disadvantages is the weakness of the systems that are designed 
according this method in forwarding functions. Since special- 
ized hardware are not used mostly in implementations, the used 
hardware is unable to handle in terms of network ports num- 
ber and expected bandwidth [3], [4], [5]. Different solutions 
have been proposed to overcome these disadvantages [6], [7], 
[8], [9] and each one provides improvements in the structure. 
Due to the mentioned issues, taking into consideration of these 
types of routers was not unexpected. Several projects in this 
area are defined dealing with the issue by various approaches. 
Basically due to the presented structure of the software routers 
and the role of a router’s components, software routers focus 
on two main mechanisms [10], [11], [12], [13], [14] which are 
discussed and analyzed separately. These two areas are: 


e Control Plane 
e Data Plane 


Click [15] router is one of the most significant software 
routers operating in the Data Plane domain which is imple- 
mented in the Linux kernel and is fully modular [16]. Most 
software routers such as Quagga [17], XORP [18] operate in 
Control Plane domain. In this paper, in addition to review and 
introduce the existing open source software routers, efforts that 
has been made in the field and the resulting improvements are 
also discussed. Subsequently in second section, the basic con- 
cept of protocols and routing algorithms topics and afterwards 
examining routers and software routers structure is presented. 
The organizing of the related works and introducing of exist- 
ing software routers, the strengths and weaknesses points and 
its features is surveyed in the third section, then, in forth sec- 
tion, the achieved improvements and projects which are pre- 
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sented in order to overcome the software routers limitations are 
discussed. The next section investigates Challenges, gaps and 
recommendations proposed in this context, and conclusions are 
presented in sixth section. 


2. Routers and Their Architectures 


In this section, the basic concept of router, routing algorithms 
and the its architecture is presented. 


2.1. Routers 


In [19], the router is defined as: ”A router is a device that 
forwards data packets between computer networks, creating an 
overlay internetwork. A router is connected to two or more 
data lines from different networks. When a data packet comes 
in one of the lines, the router reads the address information in 
the packet to determine its ultimate destination. Then, using 
information in its routing table or routing policy, it directs the 
packet to the next network on its journey. Routers perform the 
"traffic directing” functions on the Internet. A data packet is 
typically forwarded from one router to another through the net- 
works that constitute the internetwork until it reaches its desti- 
nation node.” The hardware router is mainly described in above 
definition. There is a router in the core of any network that 
connects one network to another. Therefore, a router has the 
responsibility of the packet routings across multiple networks. 
The other networks which traffics are routed toward could be ei- 
ther in proximity or miles away like in another country. Routing 
is a process to find and select a route in a network and transfer 
the data -or more precisely the traffics- from a point to another 
using a device named router. Router is a device in network 
which is used for packet forwarding from a network to another. 
Router forwards traffic based on the routing tables. In order to 
lead the traffic it should be able to read all the incoming packets 
and choose the best way to forward the packets. Typically the 
destination address is associated with the packet to indicate the 
route and the router uses this destination address to forward the 
packet. Routing components should be considered to carry out 
the process. These components are including IP routing proto- 
col which specifies the method of communicating between the 
routers, so router is allowed to share path information during 
traffic routing. By this knowledge, the router will be able to 
share routing table information, thus routing tables are updated 
by edge routers continuously. Routing algorithms are used to 
determine the routes. The last element is the routing database 
which stores the information discovered by the routing algo- 
rithms. Typically this database is only corresponding to routing 
table entries. 


2.2. IP routing protocols 


IP routing protocols can be divided into two categories: In- 
terior Gateway Protocol (IGP) and exterior gateway protocol 
(EGP) which is considered further. The router which is located 
on the edge of the network and directs the traffic across multiple 
networks needs the access to database about the network. The 
information about the network is stored in the routing tables and 
include the following: 


e Output interfaces to destination 


e Neighboring routers to learn other existing remote net- 
works 


e The best possible route for each one of these available re- 
mote networks 


e A procedure to evaluate and maintain any kind of available 
routing information 


Routers learn the available path to other remote networks in two 
ways: Dynamically using dynamic routing protocols or manu- 
ally using static routes. 


2.2.1. Static routing 

The route between source and destination is predetermined 
and all the packets should pass through this route unless the 
network administrator make changes on that. This type of rout- 
ing is not related to the network and is configured by the net- 
work administrator. The routing table is created, maintained 
and updated by the network administrator and any static route 
should be entered manually in to the configuration files of each 
router for successful connection. Static routing requires exten- 
sive planning and management. If there is many routers in the 
network many routes should be configured, In the case of a link 
failure, there is no mechanism for router to suggest an alterna- 
tive and efficient route to the packets, so packets that are sent 
to the fail link are lost. Static routing is boring and is also not 
Fault tolerance. When a change occurs in the routing backbone 
and a link goes down, the network is unable to automatically 
update the routing table, therefore the routing table should be 
manually updated by the administrator. 


2.2.2. Dynamic routing 

Dynamic routing runs the same operation as in static routing, 
except that in dynamic routing IP protocols automatically help 
routers to update their routing tables. Therefore, they are able 
to recalculation a proper route when a link goes down. Dy- 
namic routing protocols are usually used in large networks to 
reduce operational and administrative overhead caused by the 
static routing usage. These protocols are usually configured 
in routers to facilitate the information exchanges and updates 
among routers. Dynamic routing protocols allow routers to 
share the information about remote networks dynamically and 
then automatically add them to their routing tables. No network 
administrator is required. Examples of dynamic routing proto- 
cols are: Routing Information Protocol (RIP), Enhanced Inte- 
rior Gateway Protocol (EIGRP), and Open Shortest Path First 
(OSPF). As mentioned the routing protocols are divided in two 
main classes of IGP (which is responsible for routing informa- 
tion within an AS (Autonomous system) unit in a Domain) and 
EGP (which is normally used for routing information among 
two or more ASs). An IGP is divided into two subsets called 
Distance Vector protocol and Link State protocol depending on 
how these routing protocols calculate the distance. In DV pro- 
tocol the routes are announced as a vector of distance and di- 
rection. In this case, distance can be determined in metric terms 


or defined more precisely. For example RIPv1 and RIPv2 use a 
known metric as a hop count to determine the distance between 
routers. EIGRP and IGRP are using a combination of band- 
width and delay to calculate the distance between the routers, 
while these are still under the influence of DV routing protocol. 
LS protocol is another subset of IGP that creates a holistic view 
of the whole network to describe all the possible paths along the 
cost. It creates a topological database using the Shortest Path 
First (OSPF) algorithm that represents all the network’s possi- 
ble paths. Therefore, unlike the DV protocol which broadcasts 
the data, LS protocol uses multicasting. When a router uses LS 
protocol such as OSPF or IS-IS, it notifies any changes in the 
network. In this case routers do not send their entire routing ta- 
ble information and only the information required for the router 
is sent [20]. 


2.3. Routing Protocols Overview 


Routing Information Protocol (RIP): One of the most popu- 
lar IGP protocol that was developed for the first time at Berke- 
ley University of California and was adapted to use the Berke- 
ley Standard Distribution (BSD) of Linux systems, and its aim 
is transferring the network information to neighbouring routers 
just like other routing protocols. RIP is a DV and dynamic pro- 
tocol and uses hop count as a metric. Maximum hop count 
is considered to 15 and if the destination is more than 15 hops, 
RIP assumes that the destination is not available. RIP includes 3 
versions of RIPv1, RIPv2, RIPng. RIPv1 and RIPv2 are almost 
identical. The main difference between them is that the RIPv1 
uses Broadcast and is a routing protocol with class. In contrast, 
the RIPv2 uses Multicast and is known as a classless routing 
protocol. RIPv2 exploits an Authentication’s feature just unlike 
RIPv1 that operates without needing for authentication. RIPng 
is also a version which supports IPV6. OSPF: is a protocol of 
IGP type that uses LS algorithm and has been designed by the 
IETF based on open standards. OSPF is a non-private protocol 
implemented in large enterprise networks and is also a classless 
protocol that uses the area concept in scalability. The OSPF 
generates its own routing and is updated when a change occurs 
in the network topology in multicast form. This event occurs 
once every 30 minutes. The main advantage of OSPF over RIP 
is high convergence speed and scalability to implement in large 
networks. IS-IS: is an IGP & LS protocol which operates as 
the antithesis of DV protocols such as IGRP and RIP. The IS- 
IS runs the Dijkstra’s algorithms in each network to create a 
database of the network topology and determine the best path to 
the specified destination. IS-IS is an ANSI ISO protocol and is 
used in little differences compared to OSPF. Another advantage 
of the LS in comparison to DV protocols is capability of fast 
convergence and routing loops avoidance. Thus the LS routing 
protocols are able to support large internal networks. Here is 
some of the IS-IS features: 


1. The ability of flooding new information quickly 
2. Fast convergence 

3. Hierarchical routing 

4. High Scalability 


BGP: is an EGP protocol and BGP4 is the last version due to 
some revisions. BGP performs the inter-domain routing in TCP 
/ IP and is consider to use a vector path routing algorithm. The 
main role of BGP is to route the information among multiple 
AS or domain [20]. 


2.4. Routing actions separation 


As mentioned in introduction, the main function of router (as 
a network tool) in different references such as [10], [11], [12], 
[13], [14] is concentrated in two planes. 


e Control path routines (Control Plane) 
e Data path control (switching) (Data Plane) 


Also in [13] the elements of the network are divided into two 
categories: 


e Control Elements (CE) 
e Forwarding Elements (FE) 


According to this classification the proposed architecture for 
software routers should be able to cover both domains. Figure 
1 shows these boundaries in a hardware form and each one of 
the related actions has been identified and separated. 


e Run routing protocols 

e Maintain routing tables 

e Check for CLI commands 

e Increment accounting counters 
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Figure 1: The boundaries between control panel and data plane [11]. 


Basically Control Plane implements several distributed rout- 
ing protocols that are mentioned above, such as RIP, OSPF, and 
BGP. Router can achieve a local or global view to the network 
topology by using these protocols. Control Plane elements ex- 
tract its routing table based on the topology information. Con- 
trol Plane is usually implemented in software and is executed 
on a general-purpose processor. In all types of routers -both 
hardware and software- the implementation is executed in the 
same procedure. On the other hand Forwarding Element tasks 
will get easier but in the meantime the throughput and delay of 
packets are getting more important and vital. This part is re- 
sponsible for functions other than forwarding operations, such 
as fragmentation , TTL (Time To Live) decrement and recal- 
culating Checksum. However they are simply implemented in 
hardware routers by using application specific integrated circuit 
(ASICs) chips [21]. 


2.5. Hardware routers architecture 


Several hardware architecture are presented in [11]. The 
shared-memory architecture in Figure 2-a shows that the inter- 
faces have accessed to the shared memory through a common 
bus. In this architecture, a single processor has its own dedi- 
cated memory which stores the information and routing tables. 
Interfaces receive the requiring routing information through 
shared memory processors. In Figure 2-b a different kind of 
architecture is shown called cross bar datapath. Contrary to 
shared-memory, in this architecture each interface has locally 
its own memory and processor that receives routing informa- 
tion from the main processor and it is be stored in its memory 
for future use, and in case of the routing information changing, 
the main processor (general) puts the new information available 
to them. (It should be noted that the presented architectures are 
not the only architectures that have been proposed.) 
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Figure 2: General router architecture [11] (a) Shared memory. (b) Crossbar 
datapath. 


2.6. Software routers architecture 


The type and amount of data that is transferring through In- 
ternet is getting higher and more complex every day. This in- 
creasing in volume and complexity requires more powerful and 
sophisticated network equipment and this will cause in price 
increasing and complexity of the equipment. Major producing 


devices in this area are exclusive and company monopolized 
and are generally provided in a closed form. This closeness as 
said in [3], [4] is contrary to the nature of the Internet (where 
all protocols, architectures and, etc. are presented publicly). 
Meanwhile the open source movement has come to help and 
a new field has been created. Many projects formation were 
initiated, and it was resulted in several open source software 
router. Projects like XORP, Click, Quagga, BIRD, etc. was the 
outcome of these efforts. On the other hand, by decreasing per- 
sonal computers prices and performance grow presence in vari- 
ous fields and implementation of the open source routers on PC 
was caused in the formation of new generation of routers which 
are mostly known as Open Software Router, Open Router, and 
Open Source Router. A successful project like EURO which 
basically the open software/hardware systems study based on 
personal computer architecture to design a high performance 
router is its main purpose can be presented as a turning point in 
the architecture of a software router. Selecting the router archi- 
tecture based on PC architecture is the result of the following 
facts [22]: 


1. The presence of products from different manufacturers 
with low costs (due to high production volume) 

2. The existence of complete and accurate documentation for 
the PC architecture 

3. The evolution of guaranteed performance due to the pro- 
liferation of personal computers 


Fast and easy development, changing and debugging in soft- 
ware in comparison to hardware, provide an opportunity for 
software systems to participate in the network equipment dis- 
cussion. As seen in Figure 3 the open source routers can be con- 
sidered as a product of meeting and convergence of open source 
technology, personal computers development and the develop- 
ment of network technologies. There are also some products 
in this convergence that may have some of these 3 features, for 
example MikrotiK [23] can be pointed as a non-open source 
software router, or the networking software which are formed 
as an open source product and are based on network knowl- 
edge development that some useful software such as GN3 and 
Wireshark can be cited. But none of them operates in routing 
domain. 

As an open source operating system, Linux is a turning point 
in open source software. This operating system is used as a 
platform to run the other open source softwares and imple- 
menting software routers. A PC is composed of three main 
blocks: central processing unit (CPU), random access mem- 
ory (RAM) and the peripheral devices that are connected via 
chipset which provides the complex interconnects and control 
functions. As shown in Figure 4, CPU is connected to the 
chipset via the front-side bus (FSB). RAM temporarily stores 
the data for CPU and can be accessed by memory controller on 
the chipset through the memory bus (MB). The NICs are con- 
nected to the chipset via PCI shared bus. 

Briefly, a typical PC hardware is able to easily implement 
a shared-memory and shared-bus router therefore, NIC can 
send/receive the packets directly to/from the RAM and CPU 
also route them to the valid output buffer in RAM and NIC 
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Figure 3: Convergence of different technologies in the software routers forma- 
tion. 
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Figure 4: The key components of a software router on PC-based [5]. 


will fetch data packets from the RAM and will pass them on 
wires [5]. In fact, the flow of data packets on a PC-based soft- 
ware router is depicted in Figure 5. But as already mentioned 
software routers’ mainstream is Linux operating system and its 
relevant versions. In the following figure a block diagram of 
software architecture of Linux-based routers is depicted. 


As depicted in Figure 6, control plan functions are executed 
in user space while forwarding process is done within the kernel 
completely. Here is the main set of functions: packet forward- 
ing functions (main switching operations) and support func- 
tions of control plan (signalling protocols such as routing pro- 
tocols, control protocols, etc.). IP forwarding is an essential 
element in the kernel where all of the link layer, network and 
transport layers functions are realized. In recent years, inte- 
grated network support has tested various structures and refined 
development, especially those related to the packet receiving 
mechanisms. So that, it is replaced quickly from a simple in- 
terrupt architecture (which was enacted to kernel version 2.2) 
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Figure 5: Flow of packets in a PC-based software router [3]. 
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Figure 6: Block diagram of software architecture of a Linux-based PC router 


[3]. 


to a software interrupt mechanism (called SoftNet, which was 
enacted to version 4.2.21), then a modern interrupt mechanism 
[called NAPI (New API), and was approved to kernel version 
4.2.22]. SoftNet architecture improves the performance even 1f 
an interrupt-based structure is maintained, because it reduces 
the computational overhead due to background switching by 
postponing the incoming packets details using the interruption 
scheduler. Despite all these improvements, it is proven that this 
architecture is inappropriate in the medium and high rates of 
transmission packets. In fact, in the presence of a high rate of 
incoming packets a known phenomenon is resulted as ”interrupt 
livelock” which will worsen the efficiency. NAPI architecture 
is established to explicitly increase the scalability of the system 
which can handle the network interface demands with a modern 
interrupt mechanism and allows the network interfaces switch 
to a polling mode from a comparative classic interruption man- 
agement. All Linux kernel forwarding mechanism is basically 
a chain composed of three modules: A ’reception API’ which 
is responsible for handling incoming packets (NAPI), a module 
that performs the IP layer details, and finally a ’transmission 
API” that manages the forwarding operations for outgoing net- 
work interfaces [3]. Therefore, all software implementations 
based on the Linux operating system and personal computers, 
will use this structure more or less, except that the focus in Data 


Plane and Control Plane domains will cause in different ver- 
sion development and implementations. However, the essential 
points here is the communication between the user space and 
kernel components. ForCES structure [13] is a solution to this 
problem. Based on ForCES structure forwarding elements are 
basically implemented in ASIC and are responsible for packet 
sending. Control elements operate based on general-purpose 
processors and are responsible for operations such as routing 
and signalling protocols. The standard [Forwarding and Con- 
trol Element Separation (ForCES)] is defined as the interface 
between these two units and has the responsibility of data ex- 
changing among them. The presented structure in ForCES is 
compatible with Linux architecture. The structure that is cur- 
rently using on Linux systems is known as “netlink”. Netlink 
is basically plays a message exchanger role between the user 
space and kernel and also among the kernel itself. Here the 
netlink’s role is proposed as a protocol for data exchanging be- 
tween Forwarding Engine Component (FEC) and the Control 
Plane Component (CPC) and implement the ForCES consid- 
ered structure in Linux [24], [12]. Among these, there are some 
other ways for implementing and communication between Con- 
trol Path (Routing) and Data Path (Forwarding) components 
and OpenFlow protocol [9] is one of these new types of commu- 
nication, which, unlike common types of routers that the Data- 
Path and Controlpath are operated on one device, it is possible 
for these two components to act on separate handlers in a com- 
pletely detached mode. Open Flow will be discussed further in 
Section 5. Nowadays there are open source operating systems 
that can implement the IP functionalities. Specifically, network- 
ing code in the Linux kernel have been considered more mod- 
ular. IP protocol Stack structure which is implemented inde- 
pendently from the hardware has a well-defined API compared 
to hardware-depended device driver, and let IP layer work with 
most network hardware. Linux kernel network codes imple- 
ment RFC1812 [25] projects. When sanity checks such as IP 
header checksum verification are conducted, the packets that 
are not addressed to the router will be processed by routing 
function which determines the next router’s IP address to for- 
ward the packets that should be forwarded and the output inter- 
face that packets should be enqueued on for the transmission. 
The Linux kernel considers an effective structure for routing 
cache which is implemented based on a Hash table. At first the 
router quickly starts searching using a Hash to determine the 
output port and if that fails, the entire routing table (Forwarding 
Information Base or FIB) will be searched with a slower algo- 
rithm based on prefix matching this time. Dynamic FIB tables 
are filled with data transmitting routing protocols among routers 
[22]. Linux does not provide any implementation of routing 
protocols, routing operation are performed by some available 
open source software. Basically, some of these open source 
software are a set of different Daemons which is eventually led 
to fill the routing table. The focus of this software function can 
be concentrated in Data Plane and Control Plane domain ac- 
cording to the classification conducted on network components 
[13]. Thus, the establishment location will be specified in Linux 
architecture according to the focus and domain of the software 
routers and considering the Linux operating system architec- 


ture. Briefly, given the foregoing and the architecture shown 
in Figure 6, the routers that their operating domain is in Con- 
trol Plane are implemented in User Space, and those which are 
focused on Forwarding will be implemented in Kernel Space. 
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Figure 7: The reference architecture of an integrated software router based on 
Linux [26]. 


Software routers are obviously providing two types of inter- 
nal traffic routes as ’’fast” and ”slow” respectively, just like their 
family brand. Significantly the ”fast” paths are including For- 
warding, L2 and L3 chains, and these chains are elected for the 
entire data packets that only needs to be routed or switched and 
no detail and complexity of service / control Layer from their 
local router is required. In contrast, the ’slow” paths are used 
by all control applications and local service packets (such as 
OSPF Hello and LSU and the BGP keep-alives that must be 
delivered to the local IP control application). Packets that are 
moving in ”slow” path are usually called ’exception packets” 
[26]. As shown in Figure 7, delivering exception packets to the 
service and control applications is performed through known 
standard interfaces between kernel and user space called ”net- 
work socket”. 


3. Types of Software Router 


At the start of this section, those projects that has been iden- 
tified by the time of writing this paper are brought and are cat- 
egorized depending on their function and importance and then 
further we will discuss about the best-known and most widely 
used project. At the end, a comparison is presented composed 
of supported characteristics and protocols in each project. 


3.1. Software Router Classification 


If we classify the software routers, the following categories 
can be reached according to their main characteristics: 


1. Open source software routers suitable for implementation 
on the specific hardware and embedded systems. 

2. Open source software routers suitable for implementation 
and use on personal computers. They can be divided in 
two categories due to the focus and scope of action: 


(a) Open source software routers suitable for implemen- 
tation and use on personal computers, focusing on 


the Data Plane. 
(b) Open source software routers suitable for implemen- 


tation and use on personal computers, focusing on 
the Control Plane. 

3. The distributions which basically are not a specific imple- 
mentation and are in general a form of network manage- 
ment software integrations and particularly using one of 
the software routers in group 3. 

4. Essentially implements a specific protocol, and any other 
protocols are not supported. 


Table 1 can be considered as a conclusion: 














Group-1 Group-2-1 Group-2-2 Group-3 Group-4 
Tomato [27] Click [15] | Quagga/Zebra [17] Vayatta[28] Babel[29] 
OpenWRT[30] XORP{[1] DROP[26],[31] | OpenBGPd[29] 
JaiRo[32] BIRD[33] BSDRP[34] OpenOSPFD[35] 
OPERA[36] B.A.T.M.A.N[37] 

















Table 1: Classification of routers according to their attributes 


It should be noted that in the perspective of this article’s au- 
thors, all the listed items are not examples of open source soft- 
ware router and are mentioned only because some of them are 
listed as open source routers in some references. We try to fo- 
cus more on groups 2-1 and 2-2 and they will be investigated 
completely. As mentioned above, some open source software 
routers concentrate on implementing a specific protocol, in fact 
they are a free implementation of a protocol and any other pro- 
tocols are not supported. Categories such as OpenOSPFD [35] 
and OpenBGPD [38] and Babel [29] and B.A.T.M.A.N [37] can 
be cited. It should be noted that OpenBGPD and OpenOSPFD 
were developed as an alternative to Linux-based routing collec- 
tions such as Quagga, because they do not meet the require- 
ments and quality standards of OpenBSD project. There are 
many other open source routers that cannot be presented as a 
single package, because there is a set of various Daemons and 
options such as VPN and Firewall. For example Vyatta [28] and 
OPERA [36] and DROP [26], [31] (which also uses Quagga in 
routing operations) and BSDRP [34] can be cited. Vyatta is in- 
cluding those implementations that have been created by bring- 
ing together several different software. As previously men- 
tioned, these deployments actually use known open source soft- 
ware routers. Since 2006, vyatta was developed by bringing to- 
gether several softwares including, VPN, Virtual Firewall, Vir- 
tual Router and the free version got ready to use. The early 
versions was formed based on XORP but it was replaced with 
Quagga from version 4.0 onwards, and thus unlike the previ- 
ous versions Multicast capabilities was not supported any more, 
because the XORP is the only version of open source routers 
that supports Multicast. In recent version (version 6.6) this fea- 
ture has been added again. Since 2012, Brocade Communica- 
tions Systems was renamed to ’Vyatta, a Brocade Company” 
by acquiring Vyatta. Since 2013, Brocade renamed the Vyatta 
product known as ”the Vyatta Subscription Edition (VSE)” to 
’Brocade Vyatta 5400 vRouter” and was presented as a com- 
mercial product (not the open source). Also other products that 





are discussed in the meantime, as mentioned before, are suit- 
able for specific applications. This product is presented as a 
compact and light product, to get in a specific hardware rout- 
ing (as a firmware) which uses Broadcom chipset and wire- 
less feature as well (such as the Buffalo WHR-GS54S / WHR- 
HP-G54 and Linksys’ WRT54G / GL / GS). For example in 
[27], Tomato project has been introduced and its strengths has 
been known in stability and performance also suitable ameni- 
ties are mentioned. The OpenWRT project [30] is the same as 
Tomato as well. The open source JaiRo project [32] has im- 
plemented under the license of open source based on Ubuntu 
Linux Server, so it can be deployed on different x86 hardware. 
This project is mainly developing modules for phone servers, IP 
camera streaming, and media servers. The project is sponsored 
by Sabai Technology [39], which is active in wireless equip- 
ments field. Examples of JaiRo applications are mentioned in 
[40]. 


3.2. Introducing Types of Useful Software Router 


According to the proposed structure of software routers, 
many research groups (who are mostly academic) have started 
working on the software routers design. In this review, we focus 
on the useful and well-known routers that are generally imple- 
mented in the form of one or more Daemon in UNIX (Linux). 
The software routers such as XORP [18], Quagga / Zebra [17], 
BIRD [33], [41] and Click [15] are placed in this category. 
The following section presents a fairly detailed study into this 


group. 


3.2.1. Zebra 

Zebra is a routing software package that provides routing 
services based on TCP/IP and supports routing protocols such 
as RIPv1, RIPv2, OSPFv2, BGPv4 and IPV4, IPV6 protocols 
[42], [43]. Zebra can be used as a route server and route re- 
flector or be considered as a normal router as well. One of the 
Zebra’s feature is that no specific hardware (a router for exam- 
ple) is required to run. Zebra is applicable on Linux, FreeBSD, 
NetBSD, Open BSD and sun Solaris platform. A multihome 
computer (a host with multiple interfaces) can easily be config- 
ured as a router which runs the multirouting protocols through 
Zebra. Zebra system architecture that is shown in Figure 8 is 
totally different from other routing systems this means that the 
software routing in this system consists of a single process that 
provides all the routing capabilities. Zebra uses multiprocess 
approach as a software routing includes a set of routing Dae- 
mon pull together in order to create a routing table. There is 
a routing daemon in Zebra which is responsible for any rout- 
ing protocol separately, for example: ripd for RIP, ospfd for 
OSPFv2 and bgpd for BGPv4. Moreover a master daemon 
called Zebra is used to redistribute the routes among several 
routing daemons and update the kernel routing table associated 
with routing protocols [42]. These characteristics are lead to a 
modular architecture so that each protocol can be restarted inde- 
pendently without effecting any other protocols. Zebra provides 
protocol management’s database via SNMP daemon which sup- 
ports the SMUX protocol. Zebra modular architecture has some 


advantages and limitations. In this system only routing dae- 
mons associated with the current using routing protocols need 
to be implemented. New routing protocols can easily be added 
without effecting the whole system. Moreover no single host 
is required in these daemons implementation. Here the master 
daemon and routing daemons can interactive via network pro- 
tocols. Also multiple copies of a same routing daemon can run 
simultaneously. On the other hand, multiple interface configu- 
ration is required in multiple daemon. However zebra provides 
a unified user interface shell that is attached to each daemon. 
Performance is another factor which more loads are left on each 
system in comparison to single daemon in multiple daemons 
operation [42]. Notably, Control Plane is the domain of these 
routers. A recent project called Quagga has emerged as Zebra’s 
unofficial successor. Most of the Linux/BSD based on BGP- 
router which were using Zebra are switched to Quagga all over 
the world. 
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Figure 8: Structure of Zebra. 


3.2.2. Quagga 

Quagga routing package is a software routing package that 
provides the TCP/IP-based services like Zebra and supports 
RIPv1, RIPv2, RIPng, OSPFv2, OSPFv3, BGPv4, BGPv4+, 
IS-IS routing protocols and also IPV4 and IPV6 protocols [17], 
[20], [44]. In 2002, the Quagga project started working as a 
branch of GNU Zebra. All systems that were using GNU Ze- 
bra were switched to Quagga quickly. Quagga has inherited 
most of the Zebra’s characteristic and is available under the 
GPLv2 license. Quagga is supported by UNIX platforms, par- 
ticularly FreeBSD, Linux, NetBSD, OpenBSD and Solaris and 
MacOSX also works with little effort and its domain is control 
plane. In addition to support routing protocols it can set up the 
interface’s flag, interface’s address and static routes. Quagga 
has a different administration system methods which has two 
user modes. In normal mode, user can only see the system and 
in enable mode, user is able to change the system configura- 
tion [45]. Quagga has multi-process architecture, such as ze- 
bra. Each daemon has its own configuration files and terminal 
interface. Figure 9 shows the Quagga architecture that con- 
sists of a set of processes which communicate with each other 
via the IPC. Network routing protocols such as OSPF, RIP, IS- 


IS are respectively implemented in processes such ospfd, ripd, 
is-isd [22]. A daemon process called zebra plays a role as a 
central interface between the kernel forwarding plane and this 
protocol routing processes. It also has a CLI tool that allows 
the vtysh to monitor and modify the process of changing their 
configuration. Daemon related to protocols have been imple- 
mented as in Figure 9. Zebra process maintains a shadow copy 
of the terms of forwarding packets such as network interfaces 
and the current active route table and in the other case is often 
considered as the Forwarding Information Base (FIB). Kernel 
usually manages and maintain the forwarding packets. How- 
ever it is possible for Zebra to get customized in order to inter- 
act with other forwarding engines such as a specific hardware 
(if necessary). A forwarding plane manager protocol is pro- 
vided to facilitate the task. Zebra process collects the routing 
information of routing protocol processes and stores them in its 
Routing Information Base (RIB) with a shadow copy of the re- 
lated (FIB). Zebra process may also have the configured static 
route in its RIB. Zebra process is in charge for selecting the 
best route among all the available routes to the destination and 
updating FIB to be used. Moreover, information of the current 
best route may be distributed to protocol daemon. It should be 
noted that the routing processes communicate with Zebra pro- 
cess through a protocol called Zserve [29]. But Quagga has 
some weak points. For example, students of the Belgrade Uni- 
versity have evaluated the IS-IS protocol implementation and 
examined different scenarios, they have shown that some of the 
IS-IS implemented protocol is not working properly and there 
is room for this Quagga protocol improvement [46]. Ineff- 
cient BGP protocol for route server and also multiple branches 
of Quagga such as Quagga.net [22] (official ” Master” branch), 
Euro-IX, Quagga-RE, QuaggaFlow [47] and, etc. are the other 
weaknesses in this software router. 
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Figure 9: Structure of Zebra. 


3.2.3. BIRD 

BIRD is a routing daemon, which aims to have a flawless 
design and supports the routing technology used on the Internet 
nowadays, and also designing an extensible express architecture 
that the new routing protocols can be added easily [33], [41]. 
BIRD supports the routing protocols such as BGPv4, RIPv2, 


RIPng, OSPFv2, OSPFv3 and also IPV4 and IPV6 protocols 
and in addition to supporting various routing protocols [48] it 
has the following characteristics: 


e Supporting multiple routing tables 
e Supporting router advertisement for IPV6 hosts 


e Supporting virtual routing and to exchange the routes 
among different routing tables on a single host 


e Having a command-line interface to control and online 
monitoring of the daemon status 


e Soft reconfiguration (no complex online commands is re- 
quired in configuration and it has been taken care only by 
editing the configuration file and notifying BIRD to re- 
read the configuration file, therefore it will switch to the 
new configuration and restart the only protocols that need 
to be reconfigured. In fact, BIRD applies the new config- 
uration and no daemon restarting is required.) 


e Having a powerful language to route filtering 


BIRD is designed to work with Unix-like operating systems 
and is supported by Linux, FreeBSD, NetBSD and OpenBSD 
platforms [33], But due to its modular architecture it can also be 
supported by non-Unix systems. BIRD architecture is shown in 
Figure 10. As noted above, BIRD can have one or more routing 
tables which can be either synchronous or asynchronous with 
the OS kernel or other routing tables. The routing table con- 
tains a list of known protocols. Each protocol is connected to 
the routing table through two filters that can accept, reject or 
modify the routes. An export filter reviews the bypass routes 
from the routing table to the protocol and an import filter re- 
views passing routes from protocols to the routing table. When 
a routing table takes route from a protocol, it will recalculate 
the selected route and broadcasts them to all of the protocols 
that are attached to the table. Notice that although most of the 
protocols are only interested in receiving the selected route, but 
some protocols (such as pipe) receive and process all the rout- 
ing table entries (accepted by filters). Pipe protocol is used as 
a link between the routing tables that allow the routes passing 
from a first table (which is declared as the primary table) to a 
second table (which is declared as a peer table) and vice versa, 
depending on the type of filter (export filter for passing the route 
from the primary table to the peer table and import filter for 
passing the route from peer table to the primary table). Pipe 
protocol may operate in either transparent or opaque mode. In 
transparent mode the pipe protocol retransmits the entire routes 
from one table to another and maintains the attributes and the 
original source. If both export and import filters are set to the 
accept mode, both tables must have the same content. Trans- 
parent is the default mode. In Opaque mode, pipe protocol 
retransmits the optimized routes from one table to the others 
and send/receive to the other protocols via the same method 
used in the routes. Using the protocol kernel which is not ac- 
tually a real routing protocol, different routing tables may be 


connected to kernel routing tables. In fact, this protocol syn- 
chronizes the Bird routing tables with kernel operating system, 
instead of connecting to other routers on the network. 


Figure 10: BIRD architectures and components. 





One of the weaknesses in BIRD is that a separate compilation 
is required for each protocol, although it supports IPV4 and 
IPV6. So if a dual stack router implements two samples of 
BIRD (one for IP V4 and the other for IPV6) an entirely separate 
setup (configuration file, Tool, ....) is required. The lack of IS- 
IS protocol is also another weakness point of the BIRD router. 


3.2.4. XORP 

XORP is an open-source network which is proposed to de- 
velop the open-source network platforms [18], [49]. It oper- 
ates in Control Plane domain. Operating systems that support 
XORP are including FreeBSD, OpenBSD, NetBSD, Dragin fly 
BSD and also Windows operating systems. The XORP main 
target is an open platform for implementing network protocols 
and a replacement for private and closed network products. The 
XORP was released for the first time in July 2004 under the 
license of GPL and is an open-source software that can be ob- 
tained free of charge. XORP has a unique command line (CLI) 
that is used in interactive configuration and monitoring oper- 
ations. The interface has implemented a different application 
called XORPsh that can be used by several users simultane- 
ously. Since the XORP is an Open-source platform a lack of 
consistency is seen in updating, so the problem of compatibil- 
ity with different operating systems has increased [20]. Goals 
of manufacturing XORP is to design a platform which can pro- 
vide the four basic challenges in creating an open source router: 


1. Features: the Real-world routers should support many fea- 
tures including routing protocols, managing interface, and 
line management and multicast. 

2. Extensibility: every aspect of the router should be exten- 
sible, from routing protocols down to packet forwarding 
details. The route should support multiple simulationous 
extension as long as those extensions don’t conflict. The 
API between the router components should be both open 
and easy to use. 

3. Performance: XORP is not designed for core router, at 
least not initially. However the forwarding efficiency is an 


important aspect which is the purpose of any router. The 
scalability in the face of routing table size or number of 
peers is critical as well. 


4. Robustness: the real-world routers should not cause in 
crash or misroute packets. A fragile router faces an ex- 
tremely difficult deployment path. Extensibility makes ro- 
bustness even if it is very hard to achieve [1]. 


XORP divides into two subsystems. The high-level subsys- 
tem (which is called the user level) consists of relevant rout- 
ing protocols along with routing information base and supports 
processes. Low level subsystems which initially runs in the OS 
kernel and manages the forwarding path in fact, anything that 
deals with packets and it provides the APIs for accessing the 
high level. The goal is for almost all of the high level codes 
to be agnostic as to the details of the forwarding path [50], [1]. 
A multi process architecture is developed with one process for 
routing protocols plus additional process to manage, configure 
and coordinate. To enable extensibility, a novel inter-process 
communication mechanism have been designed for communi- 
cation between these modules. The mechanism is called XORP 
resource locators (XRLs) and is conceptually similar to URL. 
URL mechanisms such as redirection aid reliability and exten- 
sibility, and their human-readable nature makes them easy to 
understand and embed in scripting languages. The lower level 
uses Click modular router which is a modular extensible toolkit 
for packet processing on conventional PCs. Figure 11 depicts 
the User-level processes of the XORP routers and built-in Click 
forwarding path. User-level processes share the main character- 
istics of the XORP architecture. XORP has four core valuable 
processes such as: 


1. Router manager process which fully manages the router. 
Storing configuration data, starting other processes such as 
needed routing protocols for the configuration and restart 
the required failed processes are some the other tasks. 


2. The finder process which stores the refreshed mapping 
among applications such as the number of router interfaces 
and in particular the required IPC call to respond the re- 
quests. 


3. Routing Information Base process which receives the 
routes from routing processes and arbite the ones that must 
be routed in forwarding path or redistribute the other rout- 
ing processes. 


4. Forwarding Engine Abstraction (FEA) process that man- 
ages the forwarding path. The FEA also manages network 
interfaces and forwarding table in the router, and provides 
the information for routing processes about the interface 
properties and events [1]. 


It is notable that the XORP router is the only system which 
supports the multicast routing protocol and multicast manage- 
ment. Although XORP has implemented for UNIX but fully 
supports the Linux operating system. XORP supports RIP, 
RIPng, BGPv4+, OSPFv3, OSPFv2, IPV4, IPV6 for unicast 
routing and PIM-SM, IGMP / MLD for multicast routings. Its 
main disadvantages are in the hardware and software area and 


has been developed for a typical computer architecture. Over- 
head, CPU and delays in data reading from the computer mem- 
ory can limit the high efficiency of routing packets. Studies has 
proven to be 90% of the total delays are caused by the same 
problem [51]. 
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Figure 11: XORP Architecture [50], [1]. 


3.2.5. Click 

Click [15], [16], [52] is a software modular architecture for 
creating a flexible and configurable router and can run in both 
two Linux modes of user space and kernel space. The perfor- 
mance domain of this software router is data plane and is com- 
posed of packet processing modules called elements. These el- 
ements are independent of each other, and implement simple 
router functions such as packet classification, queuing, schedul- 
ing. To create a router configuration, the user can connect a set 
of elements according to their needs in a directed graph. Pack- 
ets move from one element to another along the edges of the 
graph. The user can also develop a configuration by imple- 
menting a new customized module using C++ and add to the 
configuration. It can be configured at run-time and no kernel 
recompile or reboot is required. As mentioned, the router ar- 
chitecture is a directed graph and the nodes are the same as 
elements. A single element represents a router processing unit. 
An edge or junction between the elements indicates a possible 
route for packets transmission and significant characteristics of 
an element are as follows: 


1. Element class: Is similar to object in object-oriented pro- 
gramming that each element has a class which determines 
its behaviour. 

2. Output and input ports: The ports are the endpoints of a 
connection between the elements. An element may be con- 
sisted of some input and output ports, which could have 
different meanings (e.g. a normal output or error output). 

3. Configuration string: Some of the elements classes may 
have additional arguments that are used in element initial- 
izing. Configuration string are containing these arguments 
(which are usually enclosed between parentheses). 


Figure 12 shows an IP router configuration. This figure has 
two network interfaces, but can easily be extended to three or 


more. As it can be observed, there are different elements with 
different classes in this configuration which are also resulted 
in different operations. For example, the strip(14) element, 
removes the first 14 bits of packets (e.g. Ethernet headers) 
then sends to CheckIPHeader(...) element. This element also 
checks for the packets validation and removes the invalid ones. 
Then the GetIPAddress(16) element copies the IP address of 
this packet for destination IP address annotation to be used by 
the other elements and in the next phase the LookupIPRoute() 
element looks up the destination of the annotation in routing 
table, selects the output and set the annotation based on the re- 
sult. The rest of the procedure is the same as the other elements. 
Some of the elements may have some more outputs. For exam- 
ple the DecIPTTL element which has two output port, checks 
if the packet TTL is expired and then sends the packet through 
its second port (which is usually connected to the ICMPError 
element) and if a TTL packet is still valid it will be reduced and 
updated by checksum and will be sent through the first output 
for the next element. One of the important features of the Click 
is providing two types of connection among elements: Push 
and Pull. On a push connection, packets start at the source ele- 
ment and are passed downstream to the destination element. On 
a pull connection, in contrast, the destination element initiates 
packet transfer: it asks the source element to return a packet, or 
a null pointer if no packet is available. This forms are imple- 
mented from transferring packets by two virtual calling func- 
tion as push and pull. Push connections are appropriate when 
packets are entered to click router inadvertently. For example 
when a packet arrives from a network device, the router must 
queue (a FIFO queue) these packets in the way they entered for 
later use. In contrast, pull connections are appropriate when 
click router needs to control the packets processing time. For 
example a router may transfer a packet only when the device 
is ready. Pull connections also model the timing decisions to 
choose the next packet. A packet scheduler in Click in an ele- 
ment with multiple outputs and inputs which are both Push and 
Pull types. This element responds a pull request via one of the 
input selection. Another important feature is that click can op- 
erate in two modes: interrupt and polling. Since Click process 
the packets in lower priority of interrupts, when the number 
of input packets increases, the processing interrupt will even- 
tually encounter all other tasks in system with starvation, that 
would ultimately lead to reduced throughput and the problem 
can result in Livelock. Click polling mode is implemented so 
that the network interfaces can operate in this mode which re- 
sults in increased performance. One of the main limitations 
of Click performance is the virtual function calls cost that is 
greatly resolved by language tool which has been implemented 
at the click. Another limitation of click is expensive process- 
ing graph creation but it can forward the packets in higher per- 
formance compared to Linux network stack [52]. In [53], an 
extended version of the Click has implemented on a cluster of 
connected computers using high-performance interconnection 
network. Based on the results, the performance is linearly is 
increased with the cluster size. 

At the end of this section all the properties and supported 
protocols in different software router is shown in Table 2. 
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Figure 12: An IP router configured with two network interface [16]. 


4. Related works and strategies for overcoming limitations 


Open source systems which has been developed based on PC 
structure have some limitations. As mentioned before one of the 
basic problems is in forwarding section with some limitation as 
follows: 


e Limited bus and Central Processing Unit (CPU) bandwidth 
e High memory access latency 


e Limited scalability in terms of number of network inter- 
face cards [4], [5] 


Some solution are presented to overcome these problems 
which is discussed in this section. One of these solutions is 
using a distributed structure and another one is using hardware 
implementation. 


4.0.6. Multi-stage architecture 

Multi-stage architecture [7], [8], [54] is a strategy which will 
present the following advantages by using a distributed struc- 
ture and the integration of several computers instead of one: 


1. Increase the performance of single-software routers 
2. Scale router size 
3. Distribute packet-forwarding and control functionalities 




































































Table 2: Properties and supported protocols in different software router. 


4. Recover from single-component failures 


5. Incrementally upgrade router performance 


This architecture is implemented in a way that will eventually 
be seen as a unified structure, and Control Plane and Data Plane 
operations are provided as a concentrated form. 
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and four 1-GigE Ethernet ports. Hardware source code descrip- 
tion (gateware) and software source code is available online and 
free to create simple designs such as a 4-port IPV4 router and 
a 4-port network card [55], [56], [57]. The traffic on Gigabit 
Ethernet link can be processed with a 4 full Gigabit/second rate 
using Netfpga platform. The located FPGA on the platform di- 
rectly handles datapath switching, routing and Ethernet packet 
processing and the Internet, and thus this software is only re- 
sponsible for the controlpath functions. Several models of the 
hardware is provided that are different in speed rate. In Table 3 
, the types and each model characteristics is shown [58]. 
NetFPGA model 


FPGA model Speed rate 





NetFPGA-SUME xilinx-Virtex-7 690T 10/100Gbps 


1Gbps 









4 
NetFPGA-1G-CML Xilinx-Kintex-7 
NetFPGA-10G Xilinx-Virtex- 1/10Gbps 
NetFPGA-1G Xilinx-Virtex-II Pro 50 1Gbps 


Interconnection network 








m) 
= 

















L3 router 


N 


$ H 


SA 






A- 
© 


Front NICs 





Figure 13: Architecture of Multi-stage [7]. 


This router architecture with distributed structure 1s com- 
posed of three different Stages as follows: 


1. First Stage: an array of load balancing (LB) which is used 
to distribute the load across routers that are located in the 
back-end stage. This Stage provides the input / output 
ports of multistage routers. It can be easily implemented 
in hardware using FPGA due to its simplicity, although we 
can took advantage of a standard PC. 

2. Interconnection network: this is a standard Ethernet net- 
work that is used to support fault recovery and upgrade the 
router’s switching capabilities by multiple paths between 
LB and back-end forwarding engine (FE). 
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Table 3: Type of NetFPGA. 


It should be noted that several projects has been presented 
using this hardware. In [59], an IP lookup scheme using prefect 
hashing and Blooming tree mapped on NetFPGA that improves 
the memory usage and number of accesses to the memory is 
presented. In [60], a prototype implementation of a 4-core net- 
work processor using the NetFPGA platform is described. The 
proposed simplified network processor can achieve throughput 
2.79 Gigabits per second for packet forwarding. In [61], [62] 
the energy consumption of the NetFPGA routing card into fine- 
grained per-packet and per Byte components for TCP streams, 
and file transfer in different packet size and packet rate is con- 
sidered. 


4.0.8. HERO (High-speed Enhanced Routing Operation) 
Recently, the costs are reduced compared to the commercial 
routers due to increasing attention on software router running 
on PCs with common PCI buses which has been located in seg- 
ments routing with multi Gigabit per second rate. Nevertheless, 
the available commercial network interface cards (NICs) are not 
programmable and not only a packet must pass twice through 


the PCI bus but packet processing also requires the operating 
system which it lowers the routing efficiency (considering Fig- 
ure 7 and the description). In [63], [64], an FPGA-based net- 
work card called HERO (High-speed Enhanced Routing Op- 
eration) is designed to destroy the limitations associated with 
commercial network cards. The HERO structure is as follows: 
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Figure 14: HERO Structure [63]. 


HERO is composed of three parts and performs the following 
actions respectively: 


1. Configure NIC through interactive with driver using regis- 
ters and interrupts. 

2. Input packets forwarding. For example packets are re- 
ceived from a network through a driver and these packets 
are stored in central memory when a slow route is used or 
in other NIC memories if a fast route is used. 

3. Output packets forwarding. For example when packets are 
received through a driver or other NICs and are sent to the 
network. 


The configuration center is associated with controlling routes 
and is consisted of register file (RF) block and interrupt gener- 
ator block. There is 64 independent 32-bit registers. Descriptor 
queues controls the FIFO (belongs to several packets with dif- 
ferent priorities) queues which contain RAM buffer addresses 
where slow path packets are stored. Incoming packets are man- 
aged by Incoming Packet Management block. This block re- 
ceives and buffers packets from Ethernet cores, and finally, dis- 
cards them if the FIFOs are too busy. It also accomplishes the 
routing and classification operation using a VOQ(Virtual Out- 
put Queuing) buffering architecture and eventually forwards the 
packets on either slow or fast routes [64]. Outgoing packets 
are managed by Outgoing Packet Management block which for- 
wards packets to the Ethernet cores. 


4.0.9. OpenFlow 

The main idea of OpenFlow [65], [9], [55], [66] is preparing 
the platform and equipment for researchers to test and run new 
protocols in campus networks. An open platform is not offered 
by switches and commercial routers usually. Commercial net- 
works routine is to restrict the standard external interfaces (e.g. 
forwarding packets only) and hiding the whole switch internal 
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flexibility as well. The structure of these devices is different 
from a vendor to another vendor and there is no unit and stan- 
dard structure where researchers can examine their new ideas. 
Also there are some limitations in software platforms in terms 
of the number of ports and performance. For example a PC 
with multiple network interfaces and a specific operating sys- 
tem which allows packets routing between the interfaces can be 
considered as a research platform but it faces some problems 
in performance. A PC cannot provide the required port num- 
ber for a campus wiring closet and the switches’ efficiency in 
packet processing is not available. (Wiring closet switches pro- 
cess over 100Gbit/s while PC is limited to 1Gbit/s). So Open- 
flow switches can be a solution to this problem. In the proposed 
structure of Openflow, objectives are pursued as follow [65]: 


e Amenable to high-performance and low-cost implementa- 
tions. 


Capable of supporting a broad range of research. 


Assured to isolate experimental traffic from production 
traffic. 


e Consistent with vendors’ need for closed platforms. 


For this purpose the Openflow [65], [66] provides a set of 
functions in the form of an open protocol to run on switches and 
routers through which the founded flow tables in switches and 
routers that are commonly in TCAM type can be planned and 
thus, one of the greatest limitations of a software router which 
is the limitation on the ports number supported by the PC is 
prevailed [65]. In practice the Openflow allows the available 
firmware provided by different manufacturers to communicate 
with other hardware without publishing the internal structure 
and therefore a strong hardware routing table which is benefit- 
ing a strong Forwarding feature can be written through a strong 
Control Plane which is provided by software routers. This com- 
bination results in a strong structure on both sides of a net- 
work device (refers to ForCES). An Openflow switch at least 
has three parts: 


1. A flow table and the action associated with each flow ele- 
ment that tells the switch how to process the flow. 

2. A secure channel which connects the switch to a remote 
process (the controller) and allows to send the commands 
and packets that are used between a controller and switch. 

3. The OpenFlow protocol that provides a standard an open 
path for the controller to communicate with the switch. 


By assigning a standard interface (the Openflow protocol) 
through which the flow table entries can be defined as exter- 
nal, the Openflow switch doesn’t require any planning by re- 
searchers. Currently this feature is supported optionally on 
some of the commercial products such as G8264 switches from 
IBM and some HP models switches. More details in [9] and 
[67]. [68] is a proposed work in this field, which has decom- 
posed the traditional functions of a router into two parts, namely 
the maintenance of the updated routing information and pack- 
ets forwarding. A structure has been designed by combining an 


Openflow-enabled switch and a Quagga software router which 
runs on a Linux-based PC that can obtain a higher function 
at forwarding rate. Such a method has low prices and limited 
FIB memory and is inappropriate for maintaining routing tables 
with over 300k entries. The switch has been used as a flexible 
forwarding engine to a high performance forwarding operations 
for most traffics. The PC as well is used as a route controller that 
is mainly responsible for the traffics which are not selected to be 
forwarded. In this article there is an Openflow controller, titled 
RouteVisor, which operates as a logic combination of Quagga 
and switch. Figure 15 shows the router architecture. The pro- 
grammable switch component operates as fast path and the PC 
as a route controller. This architecture is depended on a com- 
munication channel and an alternative route forwarding (slow 
path) which is between the switch and PC and is maintained by 
RoutVisor controller. More details are described in the article. 
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Figure 15: The router structure with OpenFlow switch [68]. (oriented dashed 
Lines (slow path)- oriented Bold lines (fast path) and the communication chan- 
nel is shown with Gray arrows.) 


Much efforts has been made on improving and efficiency in- 
creasing. For example, in [69] the software router functionality 
has been improved by breaking a forwarding table to obtain a 
faster [PLookup. In this paper, a structure is provided based 
on a multi-stage structure proposed in [70] which there the For- 
warding Table is broken into a number of smaller-sized table 
and they are sent to a distributed architecture, thus it will over- 
come to CPU limitations problem which is one the presented 
PC-based software router limitations with higher efficiency. But 
it should be noted that the distributed architecture may cause in 
a variety of problems which will be followed by some chal- 
lenges and solutions. For example, some of these problems are 
[69]: Packet reordering [71], handling of fragmented packets, 
synchronization of the forwarding tables in the back-stage, dif- 
ferent processor capabilities in the back-stage and intelligent 
load balancing in the back-end. The control protocol has to be 
designed to account for this distributed PC architecture. A con- 
trol protocol design for this architecture has been proposed in 
[72]. 
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5. Challenges and Recommendations 


Much efforts has been made so far in using and designing 
the open source software routers field. Some issues which can 
be further considered and provides for future activities will be 
discussed and adapted in this section. One of the challenging 
issues raised in all discussions relevant to network, in recent 
years, is issues related to energy consumption. As in other parts 
of the network we see projects concern about energy saving and 
friendly environment, such as [73], [74], [75], [76], [77] it is 
also the main idea in future activities of the next generation of 
open source software routers. Using the power efficiency of dif- 
ferent designs and combining them can be a good background 
to overcome the current problems. As mentioned before, one of 
the shortcomings of software routers is the failure in Forward- 
ing Element. In [10], [68] some of these methods have been 
used, but the combination of cheaper routers to perform tasks 
related to the Forwarding and using software for more com- 
plicated calculations and processes that control operations in a 
distributed manner, can be very effective and is efficient. This 
proposal, is opposite to the method used in projects such as [78] 
which special and powerful hardware is used, a 10G network 
card is used in this paper, but the overall architectures are the 
same as the previous architectures. Another area which could 
influence the software routers is intruding into the virtualization 
and distributed implementation domain. In this regard, some 
works are done such as [7], [8],[79] which has been limited to 
the specific application and the main focus was on improving 
the Forwarding defect. Another suggestion is using the struc- 
ture of open source routers for redundancy. This means that the 
router can be used for multiple software in each node, instead of 
using one router and the relations and exchanges among pack- 
ets can be controlled by a super software router. Thus both, the 
number of network ports (means reinforcing a part of Forward- 
ing) and the operational and computing power of the system 
increases, while the coefficient fault tolerance of the system in- 
creases. 


6. Conclusion 


The role and effect of routers in computer networks are very 
sensitive and important. Very extensive efforts has been made 
in this context to develop this element. In addition to the open 
source movement that aims to generally systems development, 
anew generation of routers were raised which besides resolving 
the closeness issue of hardware routers it also reduced the price 
of new generation routers, according to the executive architect. 
For this reason and due to the significance of open source soft- 
ware routers, we tried to survey the features and characteris- 
tics of existed software routers and improving strategies which 
will increase their efficiency while we were investigating the 
presented projects related to this subject. Also the pose chal- 
lenges in this field and future planning were also proposed. It is 
pointed as an ending conclusion that using open source software 
routers is very affordable and is useful due to the performed 
tests, including the [67] and the benefits that are discussed be- 
low: 


NN Soe ee 


. Diversity of software development tools 


Much faster and simpler development speed than in hard- 
ware implementation 

Much simpler troubleshooting and Debugging 

Existence of many possible manufacturers for products 
Low prices 


The increase in power of the systems due to the develop- 
ment of personal computers 


. The possibility of updating and evaluate the product per- 


formance continuously 


It seems that the open source routers is going to be embedded 
with the software defined networking and the virtualization net- 
work level techniques. 
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